What accuracy statistics really measure - Software, IEE Proceedings- [see also Software Engineering, IEE Proceedings]
نویسندگان
چکیده
The papcr aims to providc the software eqtimation research community with a better understanding of the meaning of, and relationship between, two statistics that are often used to assess the accuracy of predictive models: the mean magnitude rclative error, MMRE, and the nuniber of predictioiis within 25% of the actuals, pred(25). It is demonstrated that MMRE and pred(25) are, respectively, meawres of the spread and the kurtosis of the variable z where z = estimate/actual. Thus, z is considered to be a measure of accuracy, and statistics such as MAMRE and pred(25) to be measures of properties of the distribution of z. It is suggested that measures of the central location and skewness of z , as wcll as measures of spread and kurtosis, are necessary. Furthermore, since the distribution of z is non-normal, non-parametric measures of these properties may be needed. For this reason, boxplots of z are useful alternatives to simple summary metrics. It is also noted that the simple residuals are bettcr behaved than the z variable, and could also be used as the basis for comparing prediction systems.
منابع مشابه
What accuracy statistics really measure
This paper aims to provide the software estimation research community with a better understanding of the meaning of, and relationship between, two statistics that are often used to assess the accuracy of predictive models: the mean magnitude relative error, MMRE, and the number of predictions within 25% of the actuals, pred(25). We demonstrate that MMRE and pred(25) are, respectively, measures ...
متن کاملOptimal restart times for moments of completion time - Software, IEE Proceedings- [see also Software Engineering, IEE Proceedings]
Restart is an application-level technique that speeds up jobs with highly variable completion times. The authors present an efficient iterative algorithm to determine the restart strategy that minimises higher moments of completion time, when the total number of restarts is finite. They demonstrate its computational efficiency in comparison with alternative algorithms. They also discuss fast ap...
متن کاملTalking about OSS: Making Sense of the Bazaar: 1st Workshop on Open Source Software Engineering - Workshop Report
متن کامل
Understanding the requirements for developing open source software systems
This study presents an initial set of findings from an empirical study of social processes, technical system configurations, organizational contexts, and interrelationships that give rise to open software. The focus is directed at understanding the requirements for open software development efforts, and how the development of these requirements differs from those traditional to software enginee...
متن کامل